Fake News, Real Consequences: Recruiting Neural Networks for the Fight Against Fake News

The Fake News Challenge (FNC-1) is a public competition that aims to find automatic methods for detecting fake news. The dataset for the challenge consists of headline-body pairs, with the objective being to classify the pairs as unrelated, agreeing, disagreeing, or discussing. We developed four neural network models for FNC-1, two using a feed-forward architecture and two using a recurrent architecture. After running over 100 experiments across different model architectures and hyperparameters, we determined that the best model was a bag-of-words followed by a three-layer multi-layer perceptron (BoW MLP). The BoW MLP model achieved categorical test-set accuracy of 93%, and on the competition-specific FNC-1 metric it achieved a test-set score of 89%—a full ten percentage points above the published baseline.